1. Field of the Invention
The present invention relates to interactive methods to organize, retrieve and make effective use of large displays.
2. Description of the Related Art
Advances in technology for large displays such as Liquid Crystal Displays (LCD), plasma screen, and projectors are making large displays ubiquitous in business, education, and home environments. In the future, computerized tabletop and wall display systems will become more common. Applications will make use of the form factors of these large shared displays. Many of these applications will be different from those designed for a desktop personal computer (PC) and will require new interaction techniques.
The ‘Pond’ is a tabletop application for retrieving music Compact Disc (CD) thumbnails. Stahl, O., Wallberg, S., Soderberg, J., Humble, J., Fahl6n, L., Bullock, A., Lundberg, J. (2002) Information exploration using the Pond. Proceedings of CVE '02, pp. 72-79. The metaphor is a pond with shoals of fish. The table has a touch screen display and a Radio Frequency Identification (RFID) reader. RFID tags with pre-defined keywords are placed on the reader to initiate a query. In the Pond, a ‘group’ (shoal) of objects (fish) is retrieved and displayed in the pond; each query produces a ‘group’. There are special creel areas on the pond where the user can select and put interesting objects to make a selection shoal, where the shoal will initiate a query to bring more objects into the environment.
Tabletop applications developed at Mitsubishi Electric Research Laboratories (MERL) support the manipulation of objects and rely on territories to ‘group’ objects. The ‘Personal Digital Historian’ supports sharing personal digital data: photos, audio, video. Shen, C., Beardsley, P., Lesh, N., Moghaddam, B. (2001) Personal Digital Historian: User interface design. CHI] '01 Extended Abstracts pp. 29-30. The interaction model is a lazy Susan: users can drag their hands along the outer rim of a table to spin the displayed objects. ‘UbiTable’ and ‘DiamondSpin’ support partitioning a table into public and personal areas. Shen, C, Everitt, K. M., & Ryall, K. (2003) UbiTable: Impromptu face-to-face collaboration on horizontal interactive surfaces. Proceedings of UbiConp '03, pp. 281-288 and Shen, C., Vernier, F., Forlines, C., Ringel, M. (2004) DiamondSpin: an extensible toolkit for around the-table interaction. Proceedings of CHI '04, pp. 167-174.
Other tabletop systems with features that are related to this invention include Streitz, N. A., Geiβler, J. & Holmer, T. (1998) Roomware for cooperative buildings: Integrated design of architectural spaces and information spaces, Proceedings of CoBuild '98, Springer, pp. 4-21; Tandler, P., Prante, T. Muller-Tomfelde, C., Streitz, Steinmetz, R. (2001) ConnecTables: Dynamic coupling of displays for the flexible creation of shared workspaces. Proceedings of UIST '01, pp. I 1-20; Patten, J., Ishii, H., Hines, J., and Pangaro, G. (2001) Sensetable: A wireless object tracking platform for tangible user interfaces. Proceedings of CHI '01, pp. 253-260; Matsushita, M., Iida, M., Ohguro, T., Shirai, Y., Kakehi, Y., Naemura, T. (2004) Lumisight table: a face-to-face collaboration support system that optimizes direction of projected information to each stakeholder. Proceedings of CSCW '04, pp. 274-283; Ringel, M., Ryall, K., Shen, S., Forlines, C., Vernier, F. (2004) Release, relocate, reorient, resize: fluid techniques for document sharing on multi-user interactive tables. CHI '04 Extended Abstracts, pp. 1441-1444; Ryall, K., Forlines, C., Shen, C., Morris, M. (2004) Exploring the effects of group size and table size on interactions with tabletop shared-display groupware. Proceedings of CSCW '04. pp. 284-293; Scott, S., Sheelagh, M., Carpendale, T., Inkpen, K. (2004) Territoriality in collaborative tabletop workspaces. Proceedings of CSCW '04, pp. 294-303.
Electronic Walls and Work Surfaces
Research systems for organizing objects on electronic whiteboards and work surfaces at Palo Alto Research Center (PARC) include ‘VIKI’ and ‘Tivoli’. Shipman, F. M., Marshall, C. C., & Moran, T. P. (1995) Finding and using implicit structure in human-organized spatial information layouts. Proceedings of CHI '95, 346-353; Moran, T. P., Chiu, P., van Melle, W., Kurtenbach, G. (1995) Implicit structures for pen-based systems within a freeform interaction paradigm. Proceedings of CHI '95, pp. 487-494; Moran, T. P., van Melle, W., and Chiu, P. (1998) Spatial interpretation of domain objects integrated into a freeform electronic whiteboard. Proceedings of UIST '98, pp. 175-184. These systems support recognition of the implicit structure of objects on a freeform work surface. In particular, objects that form “blobs” are detected by determining ‘groups’ of objects that touch or by detecting ‘groups’ through clustering using spatial density.
The ‘DynaWall’ system supports grouping of electronic “cards” using a magnet metaphor. Prante, T., Streitz, N., Tandler, P. (2004) Roomware: Computers Disappear and interaction evolves. IEEE Computer, 37 (12): 47-54 (December 2004). Element cards repel each other so that when one is dragged on top of another overlap is avoided. Title cards attract other cards so that when an element card is dragged on it, they stick together. Retrieval by grouping is not supported.
Other recent research projects investigating interaction techniques for manipulation on large displays include: ‘Pick-and-Drop’; ‘Drag-and-Throw’, ‘Push-and-Throw’; ‘Drag-and-Pop’, ‘Drag-and-Pick’; ‘Push-and-Pop’. Rekimoto, J. (1997) Pick-and-Drop: A direct manipulation technique for multiple computer environments. Proceedings of UIST '97. ACM Press, pp. 31-39; Hascoet, M. (2003) Throwing models for large displays. Proceedings of HCI '03, pp. 73-77; Baudisch, P., Cutrell, E., Robbins, D., Czerwinski, M., Tandler, P. Bederson, B., and Zierlinger, A. (2003) Drag-and-Pop and Drag-and-Pick: Techniques for Accessing Remote Screen Content on Touch- and Pen-operated Systems. Proceedings of Interact '03, pp. 57-6; Collomb, M., Hascoet, M., Baudisch, P., and Lee, B. (2005) Improving drag-and-drop on wall-size displays. Proceedings of GI '05, Victoria, BC, May 2005, pp 25-32. The ‘Drag-and-Pop’ and ‘Push-and-Pop’ techniques identify potential targets by compatibility with the type of icon or document being moved. With the ‘Drag-and-Pop’ technique, as the user drags an object toward the direction of a target, copies of potential targets pop up along the direction of the drag and are brought closer to the source object. These copies are graphically connected to their originals by rubber bands. ‘Push-and-Pop’ is a similar technique, except that it surrounds the dragged object with potential targets in an overlaid “take-off area”.
Interaction and Visualization for Grouping and Retrieval
The ‘Scatter/Gather’ system supports narrowing a search by iteratively clustering the results and allowing the user to select relevant clusters; the reduced corpus based on the gathered selected clusters is re-clustered and the process repeats. Cutting, D., Karger, D., Pedersen, J., Tukey, J. (1992) Scatter/Gather: a cluster-based approach to browsing large document collections. Proc. ACM SIGIR '92, pp. 318-329.
The ‘VIBE’ system uses visualization for document retrieval with points of interest defined by key terms and a location on the display. Olsen, K. A., Korfhage, R. R., Sochats, K. M., Spring, M. B. and Williams, J. G. (1993) Visualization of a Document Collection: the VIBE System, Information Processing & Management, 29(1): 69-82. The retrieved documents are positioned based on their relevance to the points of interest using a force model. A weakness of the ‘VIBE’ type of system is that when an object is under the influence of multiple forces, it can be difficult to interpret the relevant clusters.